Evaluating deliberative dialogues focussed on healthy public policy

Background: Deliberative dialogues have recently captured attention in the public health policy arena because they have the potential to address several key factors that influence the use of research evidence in policymaking. We conducted an evaluation of three deliberative dialogues convened in Canada by the National Collaborating Centre for Healthy Public Policy in order to learn more about deliberative dialogues focussed on healthy public policy. Methods: The evaluation included a formative assessment of participants’ views about and experiences with ten key design features of the dialogues, and a summative assessment of participants’ intention to use research evidence of the type that was discussed at the dialogue. We surveyed participants immediately after each dialogue was completed and again six months later. We analyzed the ratings using descriptive statistics and the written comments by conducting a thematic analysis. Results: A total of 31 individuals participated in the three deliberative dialogues that we evaluated. The response rate was 94% (N = 29; policymakers (n = 9), stakeholders (n = 18), researchers (n = 2)) for the initial survey and 56% (n = 14) for the follow-up. All 10 of the design features that we examined as part of the formative evaluation were rated favourably by all participant groups. The findings of the summative evaluation demonstrated a mean behavioural intention score of 5.8 on a scale from 1 (strongly disagree) to 7 (strongly agree). Conclusion: Our findings reinforce the promise of deliberative dialogues as a strategy for supporting evidence-informed public health policies. Additional work is needed to understand more about which design elements work in which situations and for different issues, and whether intention to use research evidence is a suitable substitute for measuring actual behaviour change

The global stock of research evidence relevant to health systems policymaking

Background: Policymakers and stakeholders need immediate access to many types of research evidence to make informed decisions about the full range of questions that may arise regarding health systems. Methods: We examined all types of research evidence about governance, financial and delivery arrangements, and implementation strategies within health systems contained in Health Systems Evidence (HSE) (http://www.healthsystemsevidence.org). The research evidence types include evidence briefs for policy, overviews of systematic reviews, systematic reviews of effects, systematic reviews addressing other questions, systematic reviews in progress, systematic reviews being planned, economic evaluations, and health reform and health system descriptions. Specifically, we describe their distribution across health system topics and domains, trends in their production over time, availability of supplemental content in various languages, and the extent to which they focus on low- and middle-income countries (LMICs), as well as (for systematic reviews) their methodological quality and the availability of user-friendly summaries. Results: As of July 2013, HSE contained 2,629 systematic reviews of effects (of which 501 are Cochrane reviews), 614 systematic reviews addressing other questions, 283 systematic reviews in progress, 186 systematic reviews being planned, 140 review-derived products (evidence briefs and overviews of systematic reviews), 1,669 economic evaluations, 1,092 health reform descriptions, and 209 health system descriptions. Most systematic reviews address topics related to delivery arrangements (n = 2,663) or implementation strategies (n = 1,653) with far fewer addressing financial (n = 241) or governance arrangements (n = 231). In addition, 2,928 systematic reviews have been quality appraised with moderate AMSTAR ratings found for reviews addressing governance (5.6/11), financial (5.9/11), and delivery (6.3/11) arrangements and implementation strategies (6.5/11); 1,075 systematic reviews have no independently produced user-friendly summary and only 737 systematic reviews have an LMIC focus. Literature searches for half of the systematic reviews (n = 1,584, 49%) were conducted within the last five years. Conclusions: Greater effort needs to focus on assessing whether the current distribution of systematic reviews corresponds to policymakers’ and stakeholders’ priorities, updating systematic reviews, increasing the quality of systematic reviews, and focusing on LMICs

Health systems and policy research evidence in health policy making in Israel: what are researchers’ practices in transferring knowledge to policy makers?

Background: Ensuring the use of research evidence in health system management and policy decisions is an important challenge in this century. Knowledge transfer and exchange (KTE) has emerged as a paradigm to address the challenges and start closing the ‘know-do’ gap. This area of work is gaining momentum in most developed countries, yet, to date, no work has been performed in Israel within this area. The purpose of this study was to identify which KTE activities health systems and policy researchers in Israel have undertaken. Methods: A cross-sectional web-based survey of researchers who have conducted health systems and policy research in Israel was developed. The survey consisted of a demographics section, quantitative scales, and open-ended questions. The survey was sent to all health systems and policy researchers in Israel (n = 125). Results: The study response rate (28%) was relatively low as compared to other studies in the same field (range of 42% to 88%). Our survey found that more than a third of the health systems and policy researchers in Israel reported that they were frequently or always involved in the following KTE activities: interactions with target audience through the research process (i.e., during developing a research question or executing the research; 35% to 42%) or through formal or informal meetings during conferences, workshops, or conversations (40%). Less than half of the health systems and policy researchers in Israel are engaged in bridging activities aimed to facilitate target audiences to use research. Conclusions: This is a fairly new area in Israel and therefore the level of engagement of researchers in KTE activities is not very high. The low response rates could be because KTE is a new field in Israel and minimal KTE initiatives have been undertaken. It is preferable to have higher response rates, yet, after several initiatives, this was the outcome. While the findings are relevant, they may not reflect the total population of health system and policy researchers in Israel. Health system and policy researchers in Israel need to be introduced to the benefits and potential advantages of KTE in an organized and systematic way

SUPPORT Tools for evidence-informed health Policymaking (STP) 13: Preparing and using policy briefs to support evidence-informed policymaking

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers

SUPPORT Tools for evidence-informed health Policymaking (STP) 14: Organising and using policy dialogues to support evidence-informed policymaking

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers

Use of health systems and policy research evidence in the health policymaking in eastern Mediterranean countries: views and practices of researchers

<p>Abstract</p> <p>Background</p> <p>Limited research exists on researchers' knowledge transfer and exchange (KTE) in the eastern Mediterranean region (EMR). This multi-country study explores researchers' views and experiences regarding the role of health systems and policy research evidence in health policymaking in the EMR, including the factors that influence health policymaking, barriers and facilitators to the use of evidence, and the factors that increase researchers' engagement in KTE.</p> <p>Methods</p> <p>Researchers who published health systems and policy relevant research in 12 countries in the EMR (Bahrain, Egypt, Iran, Jordan, Lebanon, Libya, Morocco, Oman, Palestine, Sudan, Syria, and Yemen) were surveyed. Descriptive analysis and Linear Mixed Regression Models were performed for quantitative sections and the simple thematic analysis approach was used for open-ended questions.</p> <p>Results</p> <p>A total of 238 researchers were asked to complete the survey (response rate 56%). Researchers indicated transferring results to other researchers (67.2%) and policymakers in the government (40.5%). Less than one-quarter stated that they produced policy briefs (14.5%), disseminated messages that specified possible actions (24.4%), interacted with policymakers and stakeholders in priority-setting (16%), and involved them in their research (19.8%). Insufficient policy dialogue opportunities and collaboration between researchers and policymakers and stakeholders (67.9%), practical constraints to implementation (66%), non-receptive policy environment (61.3%), and politically sensitive findings (57.7%) hindered the use of evidence. Factors that increase researchers' engagement in KTE activities in the region were associated with involving policymakers and stakeholders at various stages such as priority-setting exercises and provision of technical assistance.</p> <p>Conclusions</p> <p>Researchers in the EMR recognize the importance of using health systems evidence in health policymaking. Potential strategies to improve the use of research evidence emphasize two-way communication between researchers and policymakers. Findings are critical for the upcoming World Health Report 2012, which will emphasize the significance of conducting and translating health research to inform health policies.</p

Towards a better understanding of the nomenclature used in information-packaging efforts to support evidence-informed policymaking in low- and middle-income countries

Background: The growing recognition of the importance of concisely communicating research evidence and other policy-relevant information to policymakers has underpinned the development of several information-packaging efforts over the past decade. This has led to a wide variability in the types of documents produced, which is at best confusing and at worst discouraging for those they intend to reach. This paper has two main objectives: to develop a better understanding of the range of documents and document names used by the organizations preparing them; and to assess whether there are any consistencies in the characteristics of sampled documents across the names employed to label (in the title) or describe (in the document or website) them. Methods: We undertook a documentary analysis of web-published document series that are prepared by a variety of organizations with the primary intention of providing information to health systems policymakers and stakeholders, and addressing questions related to health policy and health systems with a focus on low- and middle-income countries. No time limit was set. Results: In total, 109 individual documents from 24 series produced by 16 different organizations were included. The name ‘policy brief/briefing’ was the most frequently used (39%) to label or describe a document, and was used in all eight broad content areas that we identified, even though they did not have obviously common traits among them. In terms of document characteristics, most documents (90%) used skimmable formats that are easy to read, with understandable, jargon-free, language (80%). Availability of information on the methods (47%) or the quality of the presented evidence (27%) was less common. One-third (32%) chose the topic based on an explicit process to assess the demand for information from policy makers and even fewer (19%) engaged with policymakers to discuss the content of these documents such as through merit review. Conclusions: This study highlights the need for organizations embarking on future information-packaging efforts to be more thoughtful when deciding how to name these documents and the need for greater transparency in describing their content, purpose and intended audience

SUPPORT Tools for evidence-informed health Policymaking (STP)

This article is the Introduction to a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers

SUPPORT Tools for evidence-informed health Policymaking (STP) 7: Finding systematic reviews

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers

Evaluating health systems strengthening interventions in low-income and middle-income countries: are we asking the right questions?

In recent years, there have been several calls for rigorous health policy and systems research to inform efforts to strengthen health systems (HS) in low- and middle-income countries (LMICs), including the use of systems thinking concepts in designing and evaluating HS strengthening interventions. The objectives of this paper are to assess recent evaluations of HS strengthening interventions to examine the extent to which they ask a broader set of questions, and provide an appropriately comprehensive assessment of the effects of these interventions across the health system. A review of evaluations conducted in 2009-10 was performed to answer these questions. Out of 106 evaluations, less than half (43%) asked broad research questions to allow for a comprehensive assessment of the intervention's effects across multiple HS building blocks. Only half of the evaluations referred to a conceptual framework to guide their impact assessment. Overall, 24% and 9% conducted process and context evaluations, respectively, to answer the question of whether the intervention worked as intended, and if so, for whom, and under what circumstances. Almost half of the evaluations considered HS impact on one building block, while most interventions were complex targeting two or more building blocks. None incorporated evaluation designs that took into account the characteristics of complex adaptive systems such as non-linearity of effects or interactions between the HS building blocks. While we do not argue that all evaluations should be comprehensive, there is a need for more comprehensive evaluations of the wider range of the intervention's effects, when appropriate. Our findings suggest that the full range of barriers to more comprehensive evaluations need to be examined and, where appropriate, addressed. Possible barriers may include limited capacity, lack of funding, inadequate time frames, lack of demand from both researchers and research funders, or difficulties in undertaking this type of evaluatio